Camera device including privacy protection method

ABSTRACT

Camera device includes an imaging unit, a communications unit, an identified area detection unit, and an image data processing unit. The imaging unit images an area in a shooting field to obtain image data. The communications unit communicates with a memory device that stores subject information, and obtains the subject information. The identified area detection unit dynamically detects an identified area corresponding to the subject. The image data processing unit specifies a protected area in the identified area based on the subject information, and applies protection only to the protected area when the subject is a protection target. This structure provides the camera device that generates image data in which a subject to be protected is obscured in an imaged subject.

THIS APPLICATION IS A U.S. NATIONAL PHASE APPLICATION OF PCTINTERNATIONAL APPLICATION PCT/JP2008/001592, FILED ON JUN. 20, 2008.

TECHNICAL FIELD

The present invention relates to a camera device and an imaging methodof imaging a subject and recording image data of the subject that hasbeen imaged to a recording medium.

BACKGROUND ART

Digital still cameras for photographing still images; camera devices,such as digital video cameras, for taking moving images; and a range ofelectronic devices, in which a digital camera is installed, are now inwide use. This has led to a growing risk that images takenunintentionally or by chance will include subjects protected undercopyright, portrait rights, or other rights to privacy (hereinaftercollectively called “protection targets”); or that taken images or theirimage data will spread. Moreover, a function for imaging and recordingmoving images for a short period is now also provided in most digitalcameras and some camera phones. Furthermore, images captured by digitalcameras and camera phones, including subjects that require protection,can also be distributed instantaneously worldwide via the Internet.

On the other hand, the faces of people are frequently captured bymonitor cameras installed on streets and at ATM terminals, conveniencestores, public facilities, companies, factories, and so on with the aimof protecting public amenities, ensuring security, and preventing crime.These pieces of image data, created based on images captured by monitorcameras, are not always handled with consideration to privacyprotection. A range of protection technologies have therefore beenproposed to protect the images or image data of these subjects to beprotected.

One known protection technology is to obscure a subject to be protectedif it is included in a camera shooting field, by masking the subject tobe protected. In other words, portrait rights are protected by hidingprotected subjects. Likewise, a technology for making protected subjectsunidentifiable, in spite of the protected subject having been captured,is proposed in Patent Literature 1, for example.

A conventional camera has a function to obscure an image, such as byapplying a computer-generated mosaic. More specifically, theconventional camera is designed to apply a mosaic to a part of the imagein the image data obtained by photography, immediately after pressing abutton for a mosaic mode, unless an appropriate password is input. FIGS.10A and 10B show an example of image 201 taken using this type ofconventional camera. FIG. 10A is an example of image 201 taken using anormal mode without using the mosaic mode. FIG. 10B shows an example ofimage 201 taken using the mosaic mode. As shown in FIG. 10B, the usercan set area 203 for obscuring subject 202 to be protected, which is apart of image 201, when the mosaic mode is selected.

Another technology for protecting images of subjects and their imagedata is the use of a digital watermark embedded in an image so as toclearly indicate that the subject requires protection and blockphotography of this image. This technology is disclosed in PatentLiterature 2. Still another technology for blocking the photography of aprotected subject is the use of a device that transmits information onthe protected subject, such as an IC tag. This technology is disclosedin Patent Literature 3.

However, the structure of the conventional camera disclosed in PatentLiterature 1 requires the user to manually designate the image area of aprotected subject and set an area for applying protection beforestarting photography. This may result in a failure to designate itappropriately during video photography. The structure of the cameradisclosed in Patent Literature 2 and Patent Literature 3 enableprotection of the entire shooting field. However, even if a protectedsubject is just a part of the shooting field, it is not possible toapply protection to the protected subject only. In addition, theseconventional technologies cannot automatically detect protectedsubjects. Accordingly, if a protected subject moves or changes, itcannot be automatically followed.

Patent Literature 1: Japanese Patent Unexamined Publication No.2003-87632

Patent Literature 2: Japanese Patent Unexamined Publication No.2005-151124

Patent Literature 3: Japanese Patent Unexamined Publication No.2005-223601

SUMMARY OF THE INVENTION

The present invention offers a camera device and an imaging methodapplicable to the camera device for applying protection to an image areaof a subject to be protected in an image including this subject, even ifthe subject moves or changes, by following the subject.

The camera device of the present invention includes an imaging unit, acommunications unit, an identified area detection unit, and an imagedata processing unit. The imaging unit images an area in a shootingfield to obtain an image data. The communications unit obtains subjectinformation including a size, shape, direction and position of aspecific subject. The identified area detection unit dynamically detectsan identified area corresponding to the subject included in the imagedata obtained by the imaging unit. The image data processing unitspecifies a protected area in the identified area detected using theidentified area detection unit, based on the subject informationobtained by the communications unit; and applies protection only to theprotected area when the subject is a protection target to be protected.This structure provides camera device that generates image data in whicha subject requiring copyright, portrait right, or other privacyprotections is obscured in a photographed subject.

The imaging method of the present invention is a method of protecting asubject to be protected in a target on taking an image. The imagingmethod includes an imaging step, a communications step, an identifiedarea detection step, and an image data processing step. The imaging stepimages an area in the shooting field to obtain image data. Thecommunications step obtains subject information including a size, shape,direction and position of the subject. The identified area detectionstep dynamically detects an identified area corresponding to a specificsubject included in the obtained image data. The image data processingstep specifies the protected area in the detected identified area, basedon the subject information obtained; and applies protection only to theprotected area when the subject is a protection target to be protected.This offers an imaging method of generating image data in which subjectsrequiring copyright, portrait right, or other privacy protections isobscured.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating a structure of a camera device inaccordance with an exemplary embodiment of the present invention.

FIG. 2 is a block diagram illustrating a structure of a protectioncontrol unit employed in the camera device in FIG. 1.

FIG. 3 is a flow chart illustrating an imaging method applied to thecamera device in FIG. 1.

FIG. 4A is an example of protection applied to a subject to beprotected.

FIG. 4B is an example of protection applied to a subject to beprotected.

FIG. 4C is an example of protection applied to a subject to beprotected.

FIG. 4D is an example of protection applied to a subject to beprotected.

FIG. 4E is an example of protection applied to a subject to beprotected.

FIG. 4F is an example of protection applied to a subject to beprotected.

FIG. 4G is an example of protection applied to a subject to beprotected.

FIG. 4H is an example of protection applied to a subject to beprotected.

FIG. 4I is an example of protection applied to a subject to beprotected.

FIG. 5 is an example of an image generated by synthesizing image data.

FIG. 6A is an example of protection processing applied to a subject onwhich an image to be protected is displayed.

FIG. 6B is an example of protection processing applied to a subject onwhich an image to be protected is displayed.

FIG. 6C is an example of protection processing applied to a subject onwhich an image to be protected is displayed.

FIG. 7A is an example of protection processing applied to a subject onwhich an image to be protected is displayed.

FIG. 7B is an outline of a state of the camera device when a subject tobe protected is captured.

FIG. 7C is an example of protection processing applied to a subject onwhich an image to be protected is displayed.

FIG. 7D is an example of protection processing applied to a subject onwhich an image to be protected is displayed.

FIG. 8A is an example of protection processing applied to a subject onwhich an image to be protected is displayed.

FIG. 8B is an example of protection processing applied to a subject onwhich an image to be protected is displayed.

FIG. 8C is an example of protection processing applied to a subject onwhich an image to be protected is displayed.

FIG. 8D is an example of protection processing applied to a subject onwhich an image to be protected is displayed.

FIG. 9 is a flow chart illustrating a protected reproduction methodapplied to the camera device in FIG. 1.

FIG. 10A is an example of image captured by a conventional camera.

FIG. 10B is an example of image captured by the conventional camera.

REFERENCE MARKS IN THE DRAWINGS 10 Camera device 11 First camera 12Second camera 13 Autofocus processing unit 14 Network communicationsunit 15 Camera control unit 16 Image and audio forming unit 17Protection control unit 18 Protection processing unit 19Record/reproduction processing unit 20 Read and write control unit 21Output interface 22 Recording medium 23 Image and audio presentationunit 24 User interface 25 Microphone 26 Gyro unit 28 Monitor 30, 31, 42Subject 34 IC tag 35 Image 36 Image display unit 37 Art work 38 Server39 Radio waves 40 Shooting field 44 Identified area 46 Protected area 71Protection determination unit 72 Protected area control unit 73Protection accuracy control unit 74 Protected time control unit 75Protected area prediction control unit 81 Image and audio memory 82Distance estimation unit 83 Identified area detection unit 84 Digitalwatermark extractor 85 Shape searching unit

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

An exemplary embodiment of the present invention is described below withreference to drawings.

(Exemplary Embodiment)

FIG. 1 is a block diagram illustrating the structure of camera device 10in the exemplary embodiment of the present invention. FIG. 2 is a blockdiagram of the structure of protection control unit 17 employed incamera device 10.

Camera device 10 in the exemplary embodiment of the present invention isused as, for example, a digital video camera for taking moving images.Camera device 10 outputs an image after applying protection to an outputdisplay image or playback image of image data sets V10 and V20 obtainedby imaging subjects 30 and 31, when subjects 30 and 31 that are takenimages require copyright, portrait rights or other privacy protections.Camera device 10 of the present invention is also applicable to cameraphones or a digital camera linked to a range of electronic devices withdigital camera, such as camera PDAs. In addition, camera device 10 isalso applicable to digital still cameras for photographing still images,and a video-recording function provided in digital still cameras.

As shown in FIGS. 1 and 2, camera device 10 includes first camera 11(hereinafter referred to as camera 11), second camera 12 (hereinafterreferred to as camera 12), microphone 25, image and audio forming unit16, and image and audio memory 81 (hereinafter referred to as memory81). Camera device 10 also includes network communications unit 14(hereinafter referred to as communications unit 14), camera control unit15, autofocus processing unit 13 (hereinafter referred to as processingunit 13), and gyro unit 26. Still more, camera device 10 includesidentified area detection unit 83 (hereinafter referred to as detectionunit 83), shape searching unit 85, distance estimation unit 82, anddigital watermark extraction unit 84 (hereinafter referred to asextraction unit 84). Still more, camera device 10 includes protectioncontrol unit 17 and protection processing unit 18. Protection controlunit 17 includes protection determination unit 71, protected areacontrol unit 72, protection accuracy control unit 73, and protectiontime control unit 74. Furthermore, camera device 10 includesrecord/reproduction processing unit 19 (hereinafter referred to asprocessing unit 19), read and write control unit 20 (hereinafterreferred to as R/W control unit 20), output interface 21 (hereinafterreferred to as output I/F 21), image and audio presentation unit 23(hereinafter referred to as presentation unit 23), and user interface 24(hereinafter referred to as user I/F 24).

Cameras 11 and 12 are imaging unit for obtaining image data by imagingan area in a shooting field. Cameras 11 and 12 capture an image in theshooting field that includes subject 30 and subject 31, and generateimage data corresponding to the image captured. Subject 30 and subject31 are part of an imaging target. Image 35 contained in subject 30 andsubject 31 is assumed to be a specific subject that requires protection.In addition, as shown in FIG. 1, IC tag 34 is attached to subject 30.Subject 31 is, for example, image display unit 36, such as a televisionset, that displays image 35, and IC tag 34 is also attached to subject31.

Camera device 10 includes, for example, two cameras: Camera 11 andcamera 12. Twin-lens imaging is made possible by providing cameras 11and 12 with a predetermined distance in between. Cameras 11 and 12 thusenable the generation of 3-D image data. In this exemplary embodiment, astructure for obtaining a 3-D image, using two cameras 11 and 12 asshown in FIG. 1, is described. However, camera device 10 may also have astructure for displaying or recording and reproducing normal images thatare not 3-D images. For example, one of cameras 11 and 12 may be animaging unit dedicated to a distance-measuring function. Furthermore,camera device 10 may have a structure including only one of cameras 11and 12.

Cameras 11 and 12 include, for example, optical lens group (notillustrated), image pickup devices (not illustrated), and AD converter(not illustrated). The image pickup device is typically an image sensorconfigured with CCD or MOS aligned in matrix. The AD converter has afunction to convert an image signal to digital data. In cameras 11 and12, the amount of light detected by each pixel of the image pickupdevice is converted to electric signals, which in turn become imagesignals. Then, image signals are converted to image data, which aredigital signals, by an AD converter. Camera 11 outputs image data V10generated in this way. In the same way, camera 12 outputs image data V20generated. Image data V10 output from camera 11 and image data V20output from camera 12 are input to image and audio forming unit 16.

Microphone 25 captures sound around the shooting field taken by cameras11 and 12, and converts this sound to electric signals. Convertedelectric signals are input to image and audio forming unit 16 as audiodata A3 converted to digital data.

Image and audio forming unit 16 receives two pieces of image data V10and V20 output from cameras 11 and 12, respectively, and converts themto image data in a predetermined format. For example, to synthesizeimage data V30 for displaying a 3-D image, data V30 that allowsstereovision is formed from two sets of image data V10 and V20. Imageand audio forming unit 16 also receives audio data A3 output frommicrophone 25. Image and audio forming unit 16 first stores image datasets V10 and V20 and audio data A3 in memory 81. Memory 81 is a memoryfor storing image data sets V10 and V20 and audio data A3 in units offrame. Image and audio forming unit 16 stores image data sets V10 andV20 in memory 81 and extracts stored image data sets V10 and V20 asrequired. This enables the generation of image data V30, which is 3-Dimage data. Image and audio forming unit 16 synthesizes image data V30by using image data V10 output from camera 11 that corresponds to theright eye as the first field and image data V20 output from camera 12that corresponds to the left eye as the second field. An image of imagedata V30 is displayed using, for example, glasses with shutter (notillustrated) so as to switch the shutter synchrony with the timing ofthe right image data and left image data. This makes a 3-D imagevisible. Image and audio forming unit 16 supplies image data V30 toprotection processing unit 18. Image data V30 output from image andaudio forming unit 16 is not necessarily a 3-D image. It may be normalimage data V10 output from camera 11, or normal image data V20 outputfrom camera 12, or both.

Communications unit 14 is a communications means that communicates withIC tag 34 attached to subjects 30 and 31 so as to obtain tag informationstored in IC tag 34. Communications unit 14, for example, transmitshigh-frequency electromagnetic waves, such as radio waves 39, andreceives radio waves 39 such as a response signal to radio waves 39transmitted. In other words, as shown in FIG. 1, if IC tag 34 is presentnear camera device 10, IC tag 34 receives radio waves 39 transmittedfrom communications unit 14. Upon receiving radio waves 39 transmittedfrom communications unit 14, IC tag 34 responds to radio waves 39received, and transmits radio waves 30, including the tag informationstored in IC tag 34. Communications unit 14 receives radio waves 39including the tag information transmitted from IC tag 34, and extractsthe tag information from the radio signals. IC tag 34 attached tosubject 30 includes information on whether or not subject 30 to which ICtag 34 is attached requires copyright, portrait right or other privacyprotections; and subject information such as the size, shape, directionand position of subject 30. In the following description, requirementinformation refers to information on the need of copyright protectionand/or privacy protection. If subject 30 includes a protection targetthat requires copyright or privacy protection, the subject informationincludes object information on subject 30 and protection targetinformation on a protection target. In other words, the objectinformation includes object size information, object shape information,and object direction information related to subject 30. The protectiontarget information includes protection target size information,protection target shape information, protection target directioninformation, and protection target positional information. IC tag 34attached to subject 31 includes the requirement information on copyrightor privacy protection of subject 31 to which IC tag 34 is attached asthe tag information, and also includes the subject information on adisplay state of image display unit 36, such as screen size informationand screen direction information on subject 31. In the same way, ifsubject 31 includes image 35 that requires copyright or privacyprotection, the subject information includes the object information onsubject 31 and the protection target information on the protectiontarget. In other words, if the protection target is displayed on animage display screen of image display unit 36, the protection targetinformation includes screen size information and screen directioninformation on subject 31. When communications unit 14 receives the taginformation from IC tag 34 attached to subjects 30 and 31,communications unit 14 transmits this tag information to protected areacontrol unit 72.

IC tag 34 is also called an electronic tag, wireless tag, or RFID (radiofrequency identification). This tag is a small memory device with acommunications function, and an electronic circuit (not illustrated) isbuilt inside. The electronic circuit includes a memory (not illustrated)and wireless communications unit (not illustrated). The IC tag isattached to, for example, an article (not illustrated); and an IC tagdetection unit (not illustrated) reads the tag information stored in theIC tag. Reading the tag information stored in the IC tag, reveals theidentification code, owner, or identity of the article to which the ICtag is attached. In the description, as a matter of convenience,communications unit 14 communicates with IC tag 34. However, the targetthat communications unit 14 communicates to obtain the subjectinformation is not limited to IC tag 34. As long as a small memorydevice includes a memory and wireless communications unit, and cancommunicate with communications unit 14, any communications target ofcommunications unit 14 is acceptable. For example, a radiocommunications device or optical device, such as Bluetooth (Registeredtrademark of Bluetooth SIG, Inc) in which a memory is provided, and anoptical communications device is also applicable.

Subject 31 is image display unit 36 such as a television set. Imagedisplay unit 36 shown in FIG. 1 displays image 35, which typicallyrequires copyright protection, as subject 31 together with digitalwatermark for protecting image 35.

Table 1 shows an example of the tag information stored in IC tag 34. Asshown in Table 1, the tag information includes a header area and dataarea, and stores information on subjects 30 and 31 to which IC tag 34 isattached. If subjects 30 and 31 are captured by camera device 10,information on subjects 30 and 31 is recognized as the subjectinformation within a shooting field.

TABLE 1 Example of the tag information stored in IC tag 34 Item LengthNote Header area Tag-ID 8 Byte IC tag information Length 8 Byte Lengthof transmitted information Content ID 64 Byte Content ID Type 4 ByteType Encryption 4 Byte Whether encrypted or not Content ID 64 ByteContent ID Type 4 Byte Type Content Attribute VL Content attribute DataArea Content Info VL Content information Protect 4 ByteCopyright/privacy protection Target Content Size 32 Byte Objectsize/shape Content Direction 16 Byte Object direction Protect ContentSize 32 Byte Size/shape of protection target Protect Content Direction16 Byte Direction of protection target Accel 32 ByteAcceleration/angular speed of protection target Reserved VL Reservedarea Env Info VL Environment information Temp 4 Byte Target temperature/Environmental temperature Reserved VL Reserved area EPG Info VL EPGinformation Schedule VL Program schedule/Protection required or notReserved VL Reserved area CRC 4 Byte Alteration checking VL: variablelength

In the header area, “Tag-ID,” “Length,” “Content ID,” “Type,” and“Encryption” are recorded. “Tag-ID” is information on an identifier foridentifying IC tag 34, and its information length is 8 bytes. “Length”is information on a length of the tag information to be transmitted fromIC tag 34, and its information length is 8 bytes. “Content ID” isinformation for identifying subjects 30 and 31 to which IC tag 34 isattached, and its information length is 64 bytes. “Type” is contentauxiliary information of subjects 30 and 31 or information foridentifying EPG information (also called EPG info), and its informationlength is 4 bytes. “Encryption” is information on encryption that showswhether or not information recorded in the data area is encrypted, andits information length is 4 bytes.

In the data area, “Content ID,” “Type,” “Content Attribute,” and “CyclicRedundancy Check” (hereinafter referred to as CRC) are recorded.Information recorded in the data area is information to be encrypted.“Content ID” is information for identifying subjects 30 and 31 to whichIC tag 34 is attached, and its information length is 64 bytes. “Type” iscontent auxiliary information or information for identifying EPGinformation, and its information length is 4 bytes. “Content Attribute”is information on content attribute of subjects 30 and 31, and itsinformation length is variable. “CRC” is information for checkingalteration of the tag information, and its information length is 4bytes. In addition, “Content Attribute” includes “Content information”(hereinafter referred to as “Content Info”), “Environmental information”(hereinafter referred to as Env Info), and “EPG Info.” “Content Info” isinformation on contents of individual subjects 30 and 31, and itsinformation length is variable. “Env Info” is information on environmentaround subjects 30 and 31, and its information length is variable. “EPGInfo” is EPG information on image 35 contained in subject 31, and itsinformation length is variable.

“Content Info” includes “Protect,” “Target Content Size,” “ContentDirection,” “Protect Content Size,” “Protect Content Direction,”“Acceleration” (hereinafter referred to as “Accel”), and “Reserved.”“Protect” is information on whether or not subjects 30 and 31 requirecopyright or privacy protection, and its information length is 4 bytes.“Protect” also includes information on whether a protection target is apartial area or entire area, in addition to information on whether ornot subjects 30 and 31 are protection targets. Accordingly, “Protect” isindicated, for example, by “No protection” (None), “Partial protection”(Partial), and “Entire protection” (All). “Target Content Size” is theobject size information that indicates the size of subjects 30 and 31and the object shape information that indicates the shape of subjects 30and 31, and its information size is 32 bytes. The object shapeinformation may also be recorded as area information. “ContentDirection” is the object direction information that indicates thedirection of subjects 30 and 31, and its information length is 16 bytes.“Protect Content Size” is the protection target size information thatindicates the size of protection target contained in subjects 30 and 31and the protection target shape information that indicates the shape ofprotection target, and its information length is 32 bytes. Theprotection target shape information may also be recorded as the areainformation. “Protect Content Direction” is the protection targetdirection information that indicates the direction of protection targetcontained in subjects 30 and 31, and its information length is 16 bytes.“Accel” is information on the movement of protection target contained insubjects 30 and 31, and its information length is 32 bytes. “Accel” is,for example, the protection target acceleration information or theprotection target angular speed information. It is the protection targetpositional information that indicates the position of protection target.“Reserved” is a reserved area secured as a spare area for recordingcontent information further required related to subjects 30 and 31 orthe protection target contained in subjects 30 and 31, and itsinformation length is variable.

“Env Info” includes “Temp” and “Reserved.” “Temp” is environmentaltemperature information on the temperature around subjects 30 and 31 ortarget temperature information on the temperature of the protectiontarget contained in subjects 30 and 31; and its information length is 4bytes. “Reserved” is a reserved area secured as a spare area forrecording environmental information that may be further required relatedto the environment around subjects 30 and 31 or environmental conditionof the protection target itself; and its information length is variable.

“EPG Info” includes “Schedule” and “Reserved.” “Schedule” is aninformation area for recording the program schedule information relatedto an image contained in subject 31, and its information length isvariable. Furthermore, “Schedule” includes the protection requirementinformation that indicates whether or not copyright or privacyprotection is required for an image contained in subject 31. “Reserved”is a reservation area secured as a spare area for recording the EPGinformation that may be further required related to the EPG informationon the image contained in subject 31; and its information length isvariable.

These are examples of the tag information stored in IC tag 34. However,the tag information is not limited to the above structure. As long ascamera device 10 has at least the subject information including theprotection target information needed for specifying a protected area toapply protection, as described below, any structure is applicable.

Communications unit 14 can also communicate with server 39 having aserver communications unit (not illustrated). For example,communications unit 14 transmits high-frequency electromagnetic waves,such as radio waves 39, to server 38, and receives radio waves 39including a response signal, which is a response to the signaltransmitted. The response signal from server 38 is, for example,information on electronic program guide (EPG) for public broadcast thatimage display unit 36 displays. Server 38 is connected to a networkdistribution server, such as broadcast or communication contentsprovider, and can thus obtain the EPG program information from adistribution server via network. Camera control unit 15 controls cameras11 and 12 and processing unit 13 in response to imaging instructioninformation from user I/F 24. For example, when user I/F 24 notifieszoom-in information, camera control unit 15 controls the optical lensgroup of cameras 11 and 12 so as to narrow the shooting field to becaptured by cameras 11 and 12. On the other hand, if user I/F 24notifies zoom-out information, camera control unit 15 controls theoptical lens group of cameras 11 and 12 so as to widen the shootingfield to be captured by the cameras 11 and 12. Camera control unit 15notifies the information that camera control unit 15 has used forcontrolling cameras 11 and 12, such as magnification rate in case ofzoom-in, to detection unit 83 and protected area control unit 72. Thecontrol information is a part of imaging condition information thatcamera device 10 uses for taking an image.

When camera control unit 15 notifies processing unit 13 of the autofocusoperation, processing unit 13 controls the focus of the optical lensgroup of cameras 11 and 12, respectively, such that cameras 11 and 12are focused to an imaging target positioned close to the center of theshooting field. The imaging target positioned close to the center of theshooting field of cameras 11 and 12 may be subjects 30 and 31.Alternatively, processing unit 13 may also control the focus of theoptical lens group of cameras 11 and 12, respectively, so as to focus onthe imaging target in a large area. This large imaging target mayinclude subjects 30 and 31. Processing unit 13 generates distanceinformation between subjects 30 and 31 and camera device 10, andnotifies it to camera control unit 15. Camera control unit 15 notifiesthis distance information notified from processing unit 13 to detectionunit 83 and protected area control unit 72. The distance information isalso a part of the imaging condition information that camera device 10uses for taking an image.

Gyro unit 26 has a built-in gyro sensor, such as an angular speed sensoror acceleration sensor, and outputs direction information and positionalinformation of camera device 10 to which gyro unit 26 is installed,based on an output signal of the gyro sensor built in gyro unit 26. Thedirection information and the positional information output is input tocamera control unit 15. The direction information and the positionalinformation on camera device 10 input to camera control unit 15 arenotified to detection unit 83 and protected area control unit 72. Thedirection information on camera device 10 is information that indicatesthe relative direction of camera device 10. The relative direction ofcamera device 10 may be, for example, a degree of directional changefrom the start of imaging, or the relative directional relationshipbetween subjects 30 and 31 and camera device 10. In the same way, thepositional information on camera device 10 is also information thatindicates the relative position of camera device 10. The relativepositional information of camera device 10 is, for example, a degree ofpositional change from the start of imaging or the relative positionalrelationship between subjects 30 and 31 and camera device 10. Thedirection information and the positional information on camera device 10are both part of the imaging condition information that camera device 10uses for taking an image.

Detection unit 83 functions as an identified area detection means fordynamically detecting an identified area corresponding to specificsubjects 30 and 31 included in image data obtained through cameras 11and 12. In other words, detection unit 83 detects identified areascorresponding to specific subjects 30 and 31, which are protectiontargets, based on the protection target information, candidateinformation, digital watermark area information, the distanceinformation, the imaging condition information, and so on notified todetection unit 83. Detection unit 83 detects the identified area basedon basic information in the tag information, including the protectiontarget size information, the protection target shape information and theprotection target direction information on specific subjects 30 and 31;and the operation information that dynamically changes or the distanceinformation following the movement. Accordingly, the identified area canbe accurately detected even if an image is a moving image and subject30, which is a protection target, moves or changes its size. Detectionunit 83 notifies information on a detected identified area to protectedarea control unit 72 as identified area information. Detection unit 83receives the protection target information from protected area controlunit 72, the candidate information from shape searching unit 85, thedigital watermark area information from extraction unit 84, the distanceinformation from distance estimation unit 82, and the imaging conditioninformation from camera control unit 15, respectively. When detectionunit 83 dynamically detects an identified area corresponding to subjects30 and 31, the identified area does not have to be detected based on allof the protection target information, the candidate information, thedigital watermark area information, the distance information, and theimaging condition information. For example, to dynamically detect theidentified area corresponding to subject 30, the identified area can bedynamically detected based on the protection target information and thecandidate information. If a moving target in the shooting field is onlysubject 30, the identified area corresponding to subject 30 can bedynamically detected based only on the distance information or theoperation information. To dynamically detect the identified areacorresponding to subject 31, the identified area can be dynamicallydetected based only on the digital watermark area information. Themultiple uses of the protection target information, the candidateinformation, the digital watermark area information, the distanceinformation and the operation information further improve the detectionaccuracy on dynamically detecting the identified area. Shape searchingunit 85 searches a partial image area corresponding to the protectiontarget information notified from detection unit 83 in at least imagedata V10 or image data V20 stored in memory 81. In other words, if ashape like a facial shape, for example, is designated, shape searchingunit 85 searches an area where the facial shape exists in image data V10or image data V20 stored in memory 81. As a result of searching thepresence of facial shape, if shape searching unit 85 determines that thefacial shape exists, shape searching unit 85 generates positionalinformation on where the facial shape exists and size information offace. Shape searching unit 85 notifies generated positional informationand size information to detection unit 83 as the candidate information.In this way, shape searching unit 85 searches image data sets V10 andV20 for the partial image area corresponding to the protection targetshape information notified from detection unit 83. Shape searching unit85 then notifies detection unit 83 and distance estimation unit 82 ofthe size information and the positional information of a partial imagethat is searched and detected as the candidate information.

Distance estimation unit 82 estimates a distance from camera device 10to subject 30 with reference to image data V10 and image data V20. Theposition of subject 30 deviates sideways when two images in image datasets V10 and V20 of twin lenses generated by two cameras 11 and 12 areoverlaid. Closer the distance is between camera device 10 and subject30, larger the sideways deviation is in subject 30 of image data V10 andimage data V20. Accordingly, the distance from camera device 10 tosubject 30 can be estimated based on amount of sideways deviation ofsubject 30. Distance estimation unit 82 notifies detection unit 83 of adistance to subject 30 estimated based on this principle as the distanceinformation.

Extraction unit 84 refers to at least image data V10 or image data V20stored in memory 81, and determines whether or not a digital watermarkexists in the image data referred to. If extraction unit 84 determinesthat the digital watermark exists in an image of image data sets V10 andV20, extraction unit 84 generates information on image display unit 36displaying the digital watermark on an image or on area such as adisplay portion. Extraction unit 84 then notifies detection unit 83 ofthis information on area and information on this area at least includingthe size information, the shape information, and the positionalinformation as the digital watermark area information. If IC tag 34storing the tag information including information on a display state ofimage display unit 36 is attached, such as the case of subject 31,information on display state is notified from detection unit 83 toextraction unit 84 as the protection target information. Extraction unit84 uses this protection target information notified from detection unit83 for adjusting the size information and the shape information on anarea requiring copyright or privacy protection or an area to replace animage, and these pieces of information are notified to detection unit 83as the digital watermark area information.

If a protection target to apply copyright or privacy protection existsin a shooting field captured by camera device 10, protection controlunit 17 specifies a protected area in the identified area informationdetected by detection unit 83, based on information such as the taginformation obtained by communications unit 14. Protection control unit17 operates in response to an instruction signal based on userinstruction notified via user I/F 24. The instruction signal notifiedvia user I/F 24 is, for example, processing output area designationinformation, image data accuracy level designation information, orcamera operator schedule information. The camera operator scheduleinformation is, for example, output processing timing designationinformation.

As shown in FIG. 2, protection control unit 17 includes protectiondetermination unit 71, protected area control unit 72, protectionaccuracy control unit 73, and protection time control unit 74. Protectedarea control unit 72 includes protected area prediction control unit 75.

The instruction information related to imaging that is set by the user,who is an operator of camera device 10, via user I/F 24 is notified toprotection control unit 17. When the instruction information related toimaging is notified to protection control unit 17, protection controlunit 17 notifies camera control unit 15 of the instruction information.Then, when the tag information is notified from IC tag 34 tocommunications unit 14, the tag information is notified to protectioncontrol unit 17. When the tag information received by communicationsunit 14 is notified to protection control unit 17, protectiondetermination unit 71 first analyzes the tag information notified. Ifprotection determination unit 71 determines that copyright or privacyprotection is “required” in the requirement information oncopyright/privacy protection that is generated as one of analysisresults of the tag information, this determination result is notified toprotected area control unit 72, protection accuracy control unit 73, andprotection time control unit 74. Upon receiving the determination resultthat copyright or privacy protection is “required”, protected areacontrol unit 72 notifies detection unit 83 of the protection targetinformation on a protection target extracted from the tag information.Detection unit 83 analyzes a target to be protected in the shootingfield, based on the protection target information received fromprotected area control unit 72. If a target to be protected is detected,detection unit 83 notifies it to protected area control unit 72 as anidentified area. In the same way, protection accuracy control unit 73notifies the information on accuracy of protected area to protected areacontrol unit 72 so as to control the accuracy in specifying theprotected area. Protection time control unit 74 also notifiesinformation on the time to apply protection to protected area controlunit 72 so as to control the time to specify the protected area.

More specifically, protected area control unit 72 specifies a protectedarea in the identified area detected by detection unit 83. Protectedarea control unit 72 is a protected area control means for controllingan area that protection processing unit 18 applies protection. Protectedarea control unit 72 specifies the protected area in the identified areabased on the protection target information included in the taginformation notified from communications unit 14, and the imagingcondition information including the operation information, the distanceinformation and the positional information notified from camera controlunit 15. In addition, protected area control unit 72 specifies theprotected area in the identified area based on processing output areadesignation designated by the user via user I/F 24. In designating theprocessing output area, the user selects, for example, “Entire shootingfield,” “Projective area of protection target,” or “Area around theprotection target inclusive.” If “Area around the protection targetinclusive” is selected, a margin area of an accuracy-level designationvalue (e.g. 10%) set by the user is added to a boundary area of theprotection target, and thus a broader area is specified as the protectedarea. At this point, protected area control unit 72 specifies theprotected area in the identified area based on the protection accuracyinformation on the protected area notified from protection accuracycontrol unit 73. The protection accuracy information on protected areais, for example, projective area information or margin calculationinformation calculated based on a value on accuracy level designated bythe user. In other words, if “Projective area of protection target” isselected, protection accuracy control unit 73 notifies the projectivearea information to protected area control unit 72, and protected areacontrol unit 72 specifies the protected area in identified area based onthis notified projective area information. If “Area around theprotection target inclusive” is selected, protection accuracy controlunit 73 notifies the margin calculation information to protected areacontrol unit 72, and protected area control unit 72 considers thisnotified margin calculation information for specifying the protectedarea in the identified area. Protected area control unit 72 alsospecifies the protected area in identified area based on the protectiontime information on the protection time notified from protection timecontrol unit 74. Protected area control unit 72 notifies the protectedarea information on specified protected area to protection processingunit 18.

Protection accuracy control unit 73 is a protection accuracy controlmeans provided for controlling the accuracy of protection processingwhen protection is applied to the protected area. Protection accuracycontrol unit 73 calculates an area requiring protection in subjects 30and 31 as a projective area relative to the shooting field, depending onthe direction information on subjects 30 and 31 to be protected. Theprojective area information can be notified to protected area controlunit 72 as the protection accuracy information on protected area. Inaddition, protection accuracy control unit 73 calculates a margin from aboundary area of the protection target to a boundary area of protectedarea, depending on the accuracy level designation information previouslyset by the user, and notifies a margin calculation result as protectionaccuracy information to protected area control unit 72. If the user doesnot designate the accuracy level in advance, a margin area aroundsubjects 30 and 31 to be protected may be set in accordance with a stateof shooting field, i.e., the use environment of camera device 10. Inthis case, the margin area is adjusted to a predetermined area or apredetermined shape for specifying the protected area. Morespecifically, in the environment that subjects 30 and 31 includingprotection target uses easily identifiable background, the margin areacan be set small. Contrarily, in the use environment that subjects 30and 31 including protection target is not easily identifiable, themargin area may be set large. For example, it is assumed that theprotection target in subject 30 is a face, and multiple faces exist inthe shooting field. If multiple faces, which are not protection targets,larger than the face of subject 30 exist, a protection target to be atleast protected in subjects 30 and 31 is protected by setting a largermargin area.

Protection time control unit 74 is a protection time control meansprovided for controlling the time to apply protection when protection isapplied to the protected area. Protection time control unit 74 notifiesprotected area control unit 72 of the protection time information on thetime to apply protection to the protected area. This protection timeinformation is needed for specifying the protected area by protectedarea control unit 72 based on the camera operator schedule informationincluding the processing output timing designation information set bythe user via user I/F 24 or the EPG information notified fromcommunications unit 14. The protection time information includesinformation on a timing to turn on and off protection of subjects 30 and31, which are protection targets, in accordance with user setting. Theprotection time information also includes information on the time toswitch between a dynamic mask control and a static mask control relativeto protection processing of subjects 30 and 31, which are protectiontargets, in accordance with the user setting. If image display unit 36included in subject 31 is reproducing and displaying contents to beprotected, the protection time information includes information on timecorresponding to broadcast time of a specific program in which contentsto be protected is broadcast, in accordance with the EPG informationobtained by communications unit 14. Protection time control unit 74controls protected area control unit 72 such that protected area controlunit 72 specifies the protected area in the identified area based onthese pieces of the protection time information.

Protected area prediction control unit 75 is a protected area predictioncontrol means provided for predicting the movement of protected area inthe shooting field when the protected area is specified by protectedarea control unit 72, and preparing a mask area in advance. Protectedarea prediction control unit 75 predicts the movement of subject 30based on, for example, dynamically-changing specific informationnotified from detection unit 83 so as to prepare a mask area in advance.Protected area prediction control unit 75 also receives captured imagedata sets V10 and V20 from detection unit 83, and predicts the movementof protected area based on changes in frames before and after image datasets V10 and V20, so as to prepare the mask area in advance.Furthermore, protected area prediction control unit 75 can also preparethe mask area in advance by predicting the movement of protected areabased on the positional information or the direction information oncamera device 10 notified from camera control unit 15. The positionalinformation and the direction information of camera device 10 are piecesof information output from gyro unit 26. Protected area control unit 72can also specify the protected area based on the mask area informationon the protected area prepared by protected area prediction control unit75.

Protection processing unit 18 applies protection to the protected areaspecified by protection control unit 17. Protection processing unit 18replaces the protected area with mask pattern, computer-generatedmosaic, or other image; or erases the protected area so as to obscure orerase the protected area (hereinafter this type of processing is called“masking”). Protection processing unit 18 applies masking to image datacorresponding to the protected area in image data sets V10, V20, and V30so as to generate image data V31 after masking. Protection processingunit 18 supplies this image data V31 to processing unit 19 and outputI/F 21. The protected area may also be protected by masking thatdynamically replaces the protected area with another image.

In this way, protection processing unit 18 protects the protected areaspecified by protected area control unit 72, relative to image data setsV10, V20, and V30 supplied from image and audio forming unit 16,typically using computer-generated mosaic image. Protection processingunit 18 outputs an image after processing the protected area as imagedata V31. Protection control unit 17 and protection processing unit 18function as an image data processing unit for applying protection to theprotected area by specifying the protected area in the identified areadetected by detection unit 83, based on the subject information includedin the tag information obtained by communications unit 14.

Processing unit 19 functions as a recording unit for recording an imagestream, which is generated by applying compression coding to image dataV31 as image information to recording medium 22 via R/W control unit 20.Processing unit 19 reads out the image stream recorded in recordingmedium 22 via R/W control unit 20, and applies extension decoding to theimage stream read out. Image data restored by extension decoding issupplied to output I/F 21. In other words, R/W control unit 20 controlswriting to and reading from recording medium 22. Processing unit 19receives audio data A3 output from microphone 25, typically via imageand audio forming unit 16. Audio data A3 is recorded to recording medium22 and output to output I/F 21 as audio information in the same way asimage data V31. Processing unit 19 can also record and reproduce imageauxiliary information related to the image information and the audioinformation to and from recording medium 22. The image auxiliaryinformation is, for example, the subject information including theprotection target information related to image and sound. The imageauxiliary information is also the imaging condition informationcorresponding to an image taken. These pieces of the subject informationand the imaging condition information are handled as the image auxiliaryinformation separate from the image information and the audioinformation, and are recorded to and reproduced from recording medium22. The image auxiliary information recorded to recording medium 22 isnotified to protection control unit 17 via R/W control unit 20 andprocessing unit 19 when related image information is reproduced.Recording medium 22 is configured with an SD card, Blu-ray Disc(hereinafter referred to as BD), DVD, HDD, USB memory, or built-in flashmemory; and is placed in a recording medium insertion unit (notillustrated) provided on camera device 10.

Output I/F 21 is an interface for outputting image data V31 and audiodata A3 to presentation unit 23. Presentation unit 23 includes monitor28 for displaying images, such as liquid crystal display, organic ELdisplay, head-mount display, and plasma display; and audio output unit(not illustrated), such as an earphone and speaker for outputting sound.Presentation unit 23 displays an image captured by cameras 11 and 12,reproduces sound obtained by microphone 25, or reproduces an image andsound recorded in recording medium 22. Presentation unit 23 can alsoreproduce and display image data V31 in which protection is applied tothe protected area.

User I/F 24 includes an input unit such as a menu (not illustrated)displayed on a touch panel attached to a liquid crystal display unit.Via user I/F 24, the user controls imaging by camera device 10 or on andoff of audio recording. Also via user I/F 24, cameras 11 and 12 areswitched, a shooting field such as zoom-in and zoom-out is changed, andautofocus function is selected. The user can also control processingunit 19 via user I/F 24. User I/F 24 also functions as a userdesignation unit for designating the protected area by the user. Otherthan designation of the protected area, the processing output areadesignation information, accuracy level designation information of imagedata, the camera operator schedule information, and so on are also setvia user I/F 24. Each item set via user I/F 24 is notified to protectioncontrol unit 17.

As described above, camera device 10 of the present invention at leastincludes the imaging unit, the communications unit, the identified areadetection unit and the image data processing unit. The imaging unitcaptures an area in shooting field 40 so as to obtain image data. Thecommunications unit communicates with IC tag 34 storing the subjectinformation including the size, shape, direction, and position of aspecific subject, and obtains the subject information. The identifiedarea detection unit dynamically detects identified area 44 correspondingto specific subjects 30 and 31 included in image data sets V10 and V20obtained by the imaging unit. The image data processing unit specifiesprotected area 46 from identified area 44 detected by the identifiedarea detection unit and applies protection to protected area 46, basedon the subject information obtained by the communications unit, whensubjects 30 and 31 are targets to apply copyright or privacy protection.Accordingly, the present invention offers camera device that can followsubjects 30 and 31 and apply protection to an image including subjects30 and 31 to be protected so as to obscure an image area of subjects 30and 31, even if subjects 30 and 31 to be protected moves or changes.

Next describes an imaging method of generating image data V31 in whichonly an image area corresponding to subjects 30 and 31 to be protectedis protected when subjects 30 and 31 captured by camera device 10 asconfigured above require copyright, portrait right or other privacyprotections.

FIG. 3 is a flow chart illustrating a procedure for detecting identifiedarea 44 in subjects 30 and 31 to be protected in a shooting field imagedby cameras 11 and 12, and generating image data V31 by processingdetected identified area 44.

FIGS. 4A to 4I illustrate examples of applying protection to subject 30,to which IC tag 34 is attached, that requires copyright, portrait rightor privacy protection. FIGS. 4A to 4G show examples of protecting aspecific person's face, who is subject 30 to which IC tag 34 isattached. In this IC tag 34 attached to the specific person, copyrightprotection or other protection is set to “Required” in the protectionrequirement information, and also the size, shape, direction andpositional information on the specific person's face is stored as thetag information including information on a protection target. FIG. 4Ashows an image when no protection is applied within shooting field 40 ofcamera device 10. FIG. 4B shows an image when identified area, which isa candidate protection target, is detected by detection unit 83 inshooting field 40 equivalent to FIG. 4A. FIG. 4C shows an example of animage when protection is applied in shooting field 40 equivalent to FIG.4A. FIG. 4D is an image captured in shooting field 40 equivalent to FIG.4C after some time from the shooting time of FIG. 4C. FIG. 4E is animage captured in shooting field 40 equivalent to FIG. 4D still aftersome time from the shooting time of FIG. 4D. FIG. 4F is an imagecaptured in shooting field 40 equivalent to FIG. 4E still after sometime from the shooting time of FIG. 4E. FIG. 4G is an image whenprotected area 46 is specified based on a protection accuracy differentfrom that in FIG. 4D in shooting field 40 equivalent to FIG. 4D. FIG. 4His an image captured in shooting field 40 equivalent to FIG. 4G aftersome time from the shooting time of FIG. 4G. FIG. 4I is an image whenprotected area 46 is specified based on a protection accuracy differentfrom that in FIG. 4D in shooting field 40 equivalent to FIG. 4D. Asshown in FIGS. 4A to 4I, subject 30 requiring privacy protection andsubject 42 not requiring privacy protection are captured in the sameshooting field 40.

FIG. 5 is an example of an image after synthesizing twin-lens image datasets V10 and V20 stored in memory 81.

FIG. 6A shows an image when subject 31 captured in shooting field 40 isimage display unit 36 on which image 35 requiring copyright protectionis displayed. FIG. 6B shows an image when protection is applied to image35 to be protected in shooting field 40 equivalent to FIG. 6A. FIG. 6Cis an example of an image that protection is applied to image 35 to beprotected by means of a dynamic mask control.

FIG. 7A shows an image when subjects 31 captured in shooting field 40are image display unit 36 on which image 35 requiring copyrightprotection is displayed and art work 37 requiring copyright protection.FIG. 7B shows camera device 10 capturing shooting field 40 equivalent toFIG. 7A. FIG. 7C is an image taken after some time from the shootingtime of FIG. 7A. FIG. 7D is an image taken after some time from theshooting time of FIG. 7A.

FIG. 8A shows an image when subjects 31 captured in shooting field 40are image display unit 36 on which image 35 requiring copyrightprotection is displayed and art work 37 requiring copyright protection.FIG. 8B is an example of an image when protection is applied to shootingfield 40 equivalent to FIG. 8A. FIG. 8C is an example of an image whenshooting field 40 is zoomed in, relative to FIG. 8B. FIG. 8D is anexample of an image when shooting field 40 is zoomed out, relative toFIG. 8B.

Next describes the imaging method of generating image data V31 from animage captured using camera device 10, and specific examples ofprotected image data with reference to FIGS. 3 to 8.

First, as shown in FIG. 3, when the start of imaging using camera device10 is input from user I/F 24, camera device 10 starts imaging operation(Step S100). In other words, an imaging step is executed. Next,communications unit 14 waits for receiving the tag information includingthe subject information from subjects 30 and 31 (Step S101). Then, in acommunications step, protection control unit 17 controls communicationsunit 14 to transmit radio waves 39 for starting communication betweencommunications unit 14 and IC tag 34. For example, as shown in FIG. 1,if IC tag 34 exists within the reach of radio waves 39, IC tag 34responds to radio waves 39 transmitted from communications unit 14 forstarting communications, and sends the tag information to communicationsunit 14. On the other hand, if no IC tag 34 exists within the reach ofradio waves from communications unit 14, communications unit 14 receivesno response. In other words, protection control unit 17 determineswhether or not receivable data that communications unit 14 can receiveexists. If protection control unit 17 determines that communicationsunit 14 has data to receive (Yes), the operation proceeds to Step S104.If protection control unit 17 determines that there is no response (No),Step S101 is repeated until communications unit receives data (StepS102).

If IC tag 34 returns the tag information, communications unit 14notifies received tag information to protection determination unit 71.Protection determination unit 71 receives and analyzes the taginformation (Step S104). Protection determination unit 71 analyzesobtained tag information, and firstly determines the type information inthe tag information. A determination result is notified to protectiontime control unit 74 (Step S106). If the type information does notconform to the EPG information (No), the operation proceeds to a firstprotection time control step (Step S108). If the type informationconforms to the EPG information (Yes), the operation proceeds to asecond protection time control step (Step S110).

In the first protection time control step, protection time control unit74 sets a protection period based on the protection time information setby the user via user I/F 24. In other words, based on the protectiontime information set by the user via user I/F 24, protection timecontrol unit 74 notifies protected area control unit 72 of timing toapply protection to protected area 46. Then, the operation proceeds to aprotection determination step (Step S112). In the second protection timecontrol step, protection time control unit 74 sets the protection periodbased on the protection time information on airtime of a specificprogram included in the EPG information that communications unit 14obtained. In other words, a timing to apply protection to protected area46, based on the protection time information on airtime of a specificprogram included in the EPG information obtained by communications unit14, is notified from protection time control unit 74 to protected areacontrol unit 72. Then, the operation proceeds to the protectiondetermination step (Step S112).

Next, in the protection determination step, protection determinationunit 71 analyzes the tag information obtained, and determines therequirement information on copyright protection and/or privacyprotection included in the tag information. When protectiondetermination unit 71 determines that protection is “required” (Yes), inthe requirement information on copyright or privacy protection, theoperation proceeds to Step S114. If protection determination unit 71determines the protection is “not required” (No), the operation proceedsto Step S101. If the operation proceeds to Step S101, this Step S101 isrepeated until communications unit 14 receives data again.

In the protection determination step, if protection is determined to be“required,” protected area control unit 72 generates the protectiontarget information in a step of obtaining information on protectiontarget (Step S114). In other words, protected area control unit 72obtains the protection target information, including the protectiontarget size information, the protection target shape information, theprotection target direction information and the protection targetpositional information on a protection target to which IC tag 34 isattached, i.e., the protection target in subjects 30 and 31. Thisprotection target information obtained is notified to detection unit 83.At this point, protected area control unit 72 may also obtain the objectinformation including the object size information, the object shapeinformation and the object direction information on an imaging target towhich IC tag 34 is attached, i.e., subjects 30 and 31. This objectinformation obtained may also be notified to detection unit 83.

Next, in an identified area detection step, detection unit 83dynamically detects identified area 44 corresponding to specificsubjects 30 and 31 included in image data (Step S116). In this way,processing related to the tag information sent form IC tag 34 isexecuted. Cameras 11 and 12 take an image in the shooting field,including subjects 30 and 31, and image data sets V10, V20, and V30corresponding to the image for one frame, for example, are stored inmemory 81. For example, as shown in FIG. 4A, image data V10 for oneframe of an image in the shooting field, including subject 30, is storedin memory 81.

In the identified area detection step, detection unit 83 notifies shapesearching unit 85 of the protection target shape information indicatingthe shape of protection target, included in the protection targetinformation. Shape searching unit 85 searches image data sets V10, V20,and V30 stored in memory 81 for an area where partial image that meetsthe shape indicated by the protection target shape information exists inthe image corresponding to the image data. This area is notified todetection unit 83 as the candidate information. For example, if theshape of face is designated as the protection target information, shapesearching unit 85 searches for an area where facial image exists in theimage. Shape searching unit 85 notifies detection unit 83 of informationon the size, shape, direction and position of a detected partial imagethat matches the target shape as the candidate information. If shapesearching unit 85 detects multiple partial images that match the targetshape in its search, they are all notified to detection unit 83 ascandidates for protection target, respectively. Information on the size,shape, direction, and position of each of partial images (hereinafterreferred to as “candidate partial images”) is notified to detection unit83.

At this point, distance estimation unit 82 may also calculate a distancefrom camera device 10 to each candidate partial image based on thecandidate information notified from shape searching unit 85. In otherwords, distance estimation unit 82 can calculate positional deviation tothe left and right of each candidate partial image on two images, withreference to twin-lens image data V30 generated from image data sets V10and V20 stored in memory 81, and estimate the distance from cameradevice 10 to subjects 30 corresponding to each candidate partial image.For example, if three candidate partial images of face are detected, anactual distance from camera device 10 to subjects 30 and 42, which arefaces, is estimated, respectively. Distance estimation unit 82 notifiesdetection unit 83 of this calculated distance to subjects 30 and 42,corresponding to each candidate partial image, as the distanceinformation.

FIG. 5 is an example of an image synthesized from twin-lens image dataof image data sets V10 and V20 stored in memory 81. As shown in FIG. 5,closer the distance of a face is from camera device 10, larger thedeviation is between image data V10 and image data V20 in synthesizedimage. Distance estimation unit 82 utilizes this deviation width inimage data sets V10 and V20 for estimating a distance to each facecaptured, and obtains the distance information. The distance informationcan also be obtained by directly measuring a distance between cameradevice 10 and subject 30 by processing unit 13 and camera control unit15. In case of obtaining the distance information by processing unit 13and camera control unit 15, the distance information is notified todetection unit 83 and protected area control unit 72. Whether to obtainthe distance information by distance estimation unit 82 or processingunit 13 depends on setting that the user input to user I/F 24.

Detection unit 83 dynamically detects identified area 44 using a part orall of the candidate information notified from shape searching unit 85,the distance information corresponding to each candidate partial imagenotified from distance estimation unit 82, and the imaging conditioninformation notified from camera control unit 15. Detection unit 83notifies the identified area information on identified area 44, as shownin FIG. 4B, to protected area control unit 72.

Next, a presence of protected area 46 specified by the user is confirmed(Step S118). If there is user specification (Yes), user I/F 24 notifiesprotection control unit 17 of information on user-specified protectedarea 46 (Step S120). Information to be notified to protection controlunit 17 is, for example, the processing output area designationinformation and the accuracy level designation information on imagedata. If there is no user specification (No), a predetermineddesignation condition is applied to protection control unit 17.

In a protection accuracy control step, protection accuracy control unit73 notifies protected area control unit 72 of the protection accuracyinformation on protected area 46 for specifying protected area 46relative to the area information on the size, direction and position ofthe protection target (Step S122).

In the protected area control step, protected area control unit 72specifies protected area 46 relative to identified areas 44 notifiedfrom detection unit 83 (Step S124). In other words, protected areacontrol unit 72 calculates actual shape, size, direction and position ofsubjects 30 and 42 (hereinafter referred to as “candidate subjects”) inidentified areas 44 notified from detection unit 83, as shown in FIG.4B. Protected area control unit 72 applies corrective calculation to thesize of subjects 30 and 42 in identified areas 44, using a distance inthe distance information and a zoom rate in the operation information soas to calculate actual size of the candidate subject included in eachidentified area 44. Next, protected area control unit 72 compares theprotection target size information in the protection target informationand an actual size value of each candidate subject calculated so as topick out a candidate subject that has an actual size value conforming toor closest to the protection target size information. In other words,protected area control unit 72 picks out identified area 44 of thecandidate subject with the actual size value conforming to or closest tothe size of protection target in subject 30 holding IC tag 34 in thecandidate subjects in identified areas 44. In the same way, protectedarea control unit 72 may also specify protected area 46 based on theprotection target shape information, the protection target directioninformation and the protection target positional information in theprotection target information. Protected area 46 can be more accuratelyspecified by using many pieces of information as much as possible.Protected area control unit 72 notifies protection processing unit 18 ofthe information on specified protected area 46, i.e., the size, shape,position and direction of protected area 46 in the image, as theprotected area information.

In the protected area control step, protected area prediction controlunit 75 may also execute a protected area prediction control step. Ifprotected area control unit 72 specifies protected area 46, theprotected area prediction control step is the step of predicting themovement of protected area 46 in the shooting field so as to prepare amasking area in advance. In the protected area prediction control step,protected area control unit 72 may also specify the protected area basedon prepared mask area information.

Next, in a protection processing step, protection processing unit 18applies protection to protected area 46 in image data sets V10, V20, andV30 that correspond to the size, shape and position on the image and aresupplied from image and audio forming unit 16, based on the protectedarea information notified from protected area control unit 72 (StepS126). Protection processing is, for example, masking for obscuring animage using a computer-generated mosaic. Image data V31, in whichprotected area 46 is protected, is generated, and sent to processingunit 19 or output I/F 21.

By executing Step S108 to Step S126 above, protection is applied to anarea requiring copyright, portrait right or privacy protection insubjects 30 and 31, and this area is obscured as protected area 46 inshooting field 40 of the image including subjects 30 and 31 to which ICtag 34 is attached. The protected area control step and the protectionprocessing step configure an image data processing step.

Next, in a reproduction step, image data V31, in which an imageprotected by the image data processing unit is superimposed on imagedata sets V10, V20, and V30 captured by cameras 11 and 12, is sent tooutput I/F 21. Audio data A3 collected by microphone 25 is also sent tooutput I/F 21. Image data V31 and audio data A3 sent to output I/F 21are reproduced on presentation unit 23, such as monitor 28 (Step S128).

Next, in a recording step, image data V31 and audio data A3 reproducedin the reproduction step are sent to processing unit 19 in the same way.Data sent to processing unit 19 is controlled by R/W control unit 20 andis recorded on recording medium 22 (Step S129). In this step, image dataV31 is recorded on recording medium 22 as the image information. Audiodata A3 collected by microphone 25 is recorded on recording medium 22 asthe audio information. The image information and the audio informationmay also be recorded together as image and audio information. Thesubject information including the protection target information onsubjects 30 and 31, which are protection targets, captured in shootingfield 40 is also sent from protected area control unit 72 to processingunit 19. The subject information sent to processing unit 19 iscontrolled by R/W control unit 20, and recorded on recording medium 22as image auxiliary information separate from the image information andthe audio information. At the same time, the imaging conditioninformation, including the operation information and the distanceinformation, used for imaging an image by camera device 10 is alsorecorded on recording medium 22 as the image auxiliary information.

Next, protection control unit 17 determines whether the instructioninformation on completion of imaging is notified from user I/F 24. Ifthe instruction information on completion of imaging is notified (Yes),the imaging operation completes (Step S130). If the instructioninformation on completion of imaging operation is not notified (No), theoperation proceeds to Step S102 again, and the above steps arerepeatedly executed.

If face-pattern processing typically used in face recognition technologyis applied, for example, when multiple faces exist in shooting field 40,as shown in FIG. 4A, all faces in shooting field 40 are picked out. Thismakes it difficult to pick out a specific face to be protected, i.e.,only subject 30 shown in FIG. 4A. A face of subject 42, which does notrequire protection, is also picked out.

On the other hand, camera device 10 in this exemplary embodiment firstreceives the protection target information on a specific face to beprotected in subject 30, which is included in the tag information sentfrom IC tag 34. Then, as shown in FIG. 4B, detection unit 83 dynamicallydetects areas where images similar to the protection target informationexist in image data within shooting field 40. These detected images arenotified to protected area control unit 72 as identified areas 44. Forexample, as shown in FIG. 4A, three faces exist in shooting field 40.Therefore, detection unit 83 detects three facial images in shootingfield 40, as shown in FIG. 4B, and notifies them to protected areacontrol unit 72 as identified areas 44. Then, protected area controlunit 72 calculates the size information, the shape information, thepositional information and the direction information of a candidatepartial image in each of three identified areas 44 notified, and specifyprotected area 46, taking into account the protection target informationand the imaging condition information.

For example, the size, shape, direction and position of each identifiedarea 44 to be protected are calculated for each frame so as to identifyprotected area 46. For example, the size of protected area 46 iscorrected, based on a zoom rate of cameras 11 and 12, and the distanceinformation. Accordingly, as shown in FIG. 4D, the movement of subject30 is followed to change protected area 46, even if subject 30 to beprotected comes close to camera device 10 and a face area to beprotected becomes larger as time passes from that shown in FIG. 4C. Theprotection target to be protected is thus reliably protected.

Also as shown in FIG. 4E, if subject 30 to be protected turns aroundrelative to camera device 10, and no face, which is a protection target,exists in shooting field 40, protected area control unit 72 specifiesthat there is no protected area 46 in shooting field 46. In this way, ifthe direction of a protection target turns around and the protectiontarget no longer exists in shooting field 40, protected area controlunit 72 accurately specifies protected area 46 by taking into accountthe direction information, in particular, on the protection targetincluded in the subject information. As a result, there is no area to beprotected within shooting field 40, and no unnecessary protection isapplied.

Also as shown in FIG. 4F, if subject 42, which is not a protectiontarget, moves to an area between subject 30, which is a protectiontarget, and camera device 10; subject 30 is hidden behind subject 42 andthus is not captured within shooting field 40. If multiple subjects 30and 42 are partially overlaid and exist in the shooting fieldsimultaneously, protected area control unit 72 detects an overlaidportion of multiple subjects 30 and 42 and specifies protected area 46based on the protection target positional information included in thesubject information. Accordingly, only an area that requires protectionin shooting field 40 is specified as protected area 46. As a result,dynamic protection is applied only to an area that requires protectionin shooting field 40.

Also as shown in FIG. 4G, protection may be applied only to a facialportion of subject 30 to be protected in an area with the size and theshape almost equivalent to a boundary area of protection target. Inother words, if the user sets an accuracy level of protected area 46 as“projective area of protection target,” protection accuracy control unit73 controls protected area control unit 72 so as to specify a projectivearea relative to an imaged area that requires protection in subject 30,based on the direction information on subject 30 to be protected.Accordingly, as shown in FIG. 4G, protection is applied only to a facialportion of the protection target.

As shown in FIG. 4H, if subject 30 to be protected turns sidewaysrelative to camera device 10, the size and the shape of the facialportion to be protected in shooting field 40 change. In this case,protected area control unit 72 identifies protected area 46 in shootingfield 40 corresponding to the direction of subject 30. If the target tobe protected changes its direction, and therefore an area of protectiontarget changes in shooting field 40, protected area control unit 72accurately identifies protected area 46 by taking into account theprotection target direction information, in particular, in the subjectinformation. As a result, as shown in FIG. 4H, protection is appliedonly to a profile image of the facial portion, which is the protectiontarget.

As shown in FIG. 4I, protection may also be applied to entire shootingfield 40 including subject 30 to be protected as protected area 46. Morespecifically, if the user sets the accuracy level of protected area 46to “Entire shooting field,” protection accuracy control unit 73 controlsprotected area control unit 72 to specify entire shooting field 40including subject 30 to be protected as protected area 46. Accordingly,as shown in FIG. 4I, protection is applied to entire shooting field 40including the facial portion of subject 30 to be protected.

As shown in FIG. 6A, if subject 31 captured in shooting field 40 isimage display unit 36 displaying image 35 to apply copyright protection,image 35 to be protected is specified as protected area 46, andprotection is applied, as shown in FIG. 6B. FIGS. 6A to 6C are the casesthat the type information does not conform to EPG information in thetype determination described in Step S106 in FIG. 3. In this case, theuser sets time to apply protection to subject 31 to be protected viauser I/F 24. Protection time control unit 74 controls protected areacontrol unit 72 based on the protection time information on protectionapplication time set by the user via user I/F 24. Accordingly, after theprotection application time set by the user passes, protection of image35 is turned off, as shown in FIG. 6A.

When the user sets time for applying protection to subject 31 to beprotected, using user I/F 24, the dynamic mask control and the staticmask control can also be switched depending on time. In this case,during the time when the static mask control is set, protection based onthe static mask control, such as computer-generated mosaic, is applied,as shown in FIG. 6B. During the time when the dynamic mask control isset, protection based on the dynamic mask control, typically a movingimage, is applied, as shown in FIG. 6C. In other words, protection timecontrol unit 74 controls protected area control unit 72 such thatprotected area control unit 72 applies protection to protected area 46in subject 31 to be protected by switching between the dynamic maskcontrol and the static mask control depending on time.

When subject 31 captured within shooting field 40 includes image 35, inwhich digital watermark is embedded, as shown in FIG. 6A, extractionunit 84 can also identify image display unit 36 or an area such asdisplay portion, on which digital watermark is embedded, in the image.In this case, the position, size and shape of the area on the imagedetermined by extraction unit 84 is notified to detection unit 83 as thewatermark area information. Based on this watermark area information,detection unit 83 dynamically detects identified area 44, and notifiesit to protected area control unit 72. Protected area control unit 72then specifies protected area 46 from identified area 44 notified, andnotifies this protected area 46 to protection processing unit 18. Thisenables protection processing unit 18 to apply protection to area wherea digital watermark is detected on the image, as shown in FIG. 6B.

As described above, camera device 10 can apply protection to an image tobe protected and obscure the image even if an image that requirescopyright, portrait right or privacy protection is displayed on imagedisplay unit 36, such as a television set, and the user takes the imagewithout being conscious of the image that requires protection. Inaddition, since protection is applied only to image display unit 36,where image 35 to be protected is displayed, an area that the userintended to photograph is not eaten away.

As shown in FIG. 7A, if subject 31 captured in shooting field 40 isimage display unit 36 displaying image 35 to apply copyright protectionand art work 37 to apply copyright protection, both image 35 to beprotected and art work 37 to be protected are identified as protectedareas 46, and protection is applied to these areas, as shown in FIG. 7B.In FIG. 7B, camera device 10 has monitor 28, which is presentation unit23, and thus image data V31 after applying protection is visible onmonitor 28.

FIG. 7C illustrates an image after protection is applied when cameradevice 10 is operated in line with a movement of the user capturingstationary subject 31. The movement of camera device 10 is detected bygyro unit 26, and the position information and the direction informationof camera device 10 are notified from gyro unit 26 to detection unit 83and protected area control unit 72. Protected area control unit 72changes protected area 46 based on a movement of camera device 10.Protected area prediction control unit 75 may also predict the shape,size, direction and position of protected area 46 that relatively moves,based on the position information and the direction information oncamera device 10 notified from gyro unit 26 to protected area controlunit 72, so as to prepare a masking area. This prepared mask area isspecified as protected area 46, and may also be notified to protectionprocessing unit 18. This enables prediction of the movement of aprotection target in shooting field 40. Accordingly, protected area 46is dynamically specified, making feasible accurate protection ofprotected area 46.

FIG. 7D illustrates an image after protection is applied when cameradevice 10 moves in line with the movement of the user imaging stationarysubject 31, same as in FIG. 7C. FIG. 7D shows an image in shooting fieldwhen the user moves to the back of image display unit 36. Since imagedisplay unit 36 is present between art work 37 and camera device 10, apart of art work 37 to be protected overlaps with image display unit 36.Image 35 to be protected displayed on image display unit 36 does notexist in shooting field 40 because the user has moved to the back ofimage display unit 36. When image 35 to be protected no longer exists inshooting field 40, as is in this example, protected area control unit 72specifies protected area 46 such that no protected area 46 correspondingto image 35 exists within shooting field 40. In this way, when adirection of a target to be protected changes, and the target to beprotected no longer exists within shooting field 40, protected areacontrol unit 72 accurately specifies protected area 46 by taking intoaccount the direction information, in particular, on the protectiontarget included in the subject information. As a result, an area toapply protection does not exist in shooting field 40, and thus nounnecessary protection is applied. Furthermore, if a part of multiplesubjects 31 in shooting field 40 exists in an overlapped state,protected area control unit 72 detects an overlapped portion of multiplesubjects 31 based on the protection target positional information, inparticular, included in the subject information. This enables a dynamicchange of protected area 46 for protecting art work 37. As a result,only an area requiring protection in shooting field 40 is dynamicallyprotected.

As shown in FIG. 8A, if subject 31 captured in shooting field 40 isimage display unit 36 displaying image 35 to apply copyright protectionand art work 37 to apply copyright protection, image 35 to be protectedand art work 37 to be protected are identified as protected areas 46,and protection is applied, as shown in FIG. 8B. If the user zooms incamera device 10, the size of protection target in shooting field 40changes, as shown in FIG. 8C. If a magnification rate of camera device10 changes by zoom-in or zoom-out operation, camera control unit 15notifies protected area control unit 72 of information on a change inmagnification rate as the operation information. Protected area controlunit 72 then changes protected area 46, based on the operationinformation notified, and notifies protection processing unit 18 ofchanged protected area information. Accordingly, protected area 46 ischanged following the movement of protection target in shooting field40, as shown in FIG. 8C, when camera device 10 executes the zoom-inoperation. In the same way, when camera device 10 executes the zoom-outoperation, protected area 46 is changed following the movement of theprotection target in shooting field, as shown in FIG. 8D.

As shown in FIGS. 8A to 8D, image display unit 36 captured in shootingfield 40 reproduces and displays a content to be protected. In addition,image display unit 36 holds the EPG information, and the EPG informationon contents to be displayed on image display unit 36 is stored in IC tag34 attached to image display unit 36. In other words, in the typedetermination described in Step S106 in FIG. 3, the type informationincluded in the protection target information stored in IC tag 34attached to image display unit 36 captured in shooting field 40 conformsto EPG information. In this case, protection time control unit 74obtains the EPG information via communications unit 14, and controlsprotected area control unit 72 such that it identifies protected area 46only during the time assumed to be a broadcast time of a specificprogram, based on the EPG information. Only during this time, protectionis applied to a content to be protected, which is reproduced on imagedisplay unit 36, according to the EPG information obtained. Accordingly,protection processing unit 18 can apply protection only to a specificprogram to be protected. As a result, protection is applied to aprotection target only when required.

As described above, the imaging method applied to camera device 10 ofthe present invention is the method of imaging and applying protectionto subjects 30 and 31 that require copyright or privacy protectioncontained in a protection target. This imaging method of the presentinvention at least includes the imaging step, the communications step,the identified area detection step and the image data processing step.The imaging step is for imaging an area in shooting field 40 so as toobtain image data. The communications step is for communicating with ICtag 34 storing the subject information including the size, shape,direction and position of a specific subject, so as to obtain thesubject information. The identified area detection step is fordynamically detecting identified area 44 corresponding to specificsubjects 30 and 31 including image data sets V10 and V20 obtained in theimaging step. The image data processing step is for specifying protectedarea 46 from identified area 44 detected in the identified areadetection step, based on the subject information obtained in thecommunications step if subjects 30 and 31 requires copyright or privacyprotection. By executing the imaging step to the image data protectionprocessing step, protected area 46 is dynamically specified for applyingcopyright, portrait right or privacy protection to subjects 30 and 31 ofan image in shooting field 40 including subjects 30 and 31 where IC tag34 is attached. In particular, for example, identified area 44 thatbecomes a protection target in a moving image is followed and protectedso as to calculate for each image frame the position and size ofprotected area 46 and protect it. In addition, the size of a target tobe protected in a moving image is accurately corrected based on theoperation information of camera device 10 and the distance informationbetween camera device 10 and subjects 30 and 31. Accordingly, even ifsubjects 30 and 31 to be protected move, camera device 10 is moved, orthe size of subject in the image changes due to a change inmagnification rate; protection is applicable to the subject,corresponding to a change in its size.

This exemplary embodiment refers to a structure of recording image dataV31 including protected area 46, to which protection is applied, inrecording medium 22. However, a structure to display image data setsV10, V20, and V30 that are not protected on presentation unit 23 is alsoapplicable. In other words, image data recorded to recording medium 22and image data read out from recording medium 22 and displayed onpresentation unit 23 may be changed as required and supplied as imagedata V31 including protected area 46 after applying protection and imagedata V10, V20, and V30 to which protection is not applied.

Still more, this exemplary embodiment refers to a structure of detectingidentified areas 44 corresponding to subjects 30 and 31 to applycopyright or privacy protection, and specifying protected area 46 fromidentified areas 44 so as to apply protection. However, if the userattaches IC tag 34 to subjects 30 and 31 to be deleted from the image inadvance, a structure including an edit function to replace protectedareas 46 corresponding to subjects 30 and 31 to be deleted with otherimages, respectively, is also applicable.

Still more, as shown in FIG. 3, this exemplary embodiment refers to astructure of specifying protected area 46 so as to apply protection atthe time of imaging subjects 30 and 31 including a protection target byusing cameras 11 and 12 in camera device 10. However, the presentinvention is not limited to the operation of dynamically detectingidentified area 44, specifying protected area 46 from identified areas44, and applying protection to specified protected area 46 at the sametime as capturing an image. For example, in case of imaging subjects 30and 31 including a protection target, for example, captured image datasets V10, V20, and V30, and audio data A3 are recorded as image andaudio information on recoding media 22 via processing unit 19. Inrecording the image and audio information, the subject informationincluding the protection target information on subjects 30 and 31 to becaptured is recorded on recording medium 22 as the image auxiliaryinformation. At the same time, the imaging condition informationincluding the operation information and the distance information relatedto imaging of an image by camera device 10 is recorded on recordingmedium 22 as the image auxiliary information. At the time of reproducingthe image and audio information, identified area 44 may be dynamicallydetected, protected area 46 may be specified from identified area 44,and protection may be applied to specified protected area 46. FIG. 9 isa flow chart illustrating protected reproduction method for applyingprotection to an image to be protected at the time of reproducing thistype of image data. In the flow chart showing the protected reproductionmethod in FIG. 9, the step names same as those in the flow chart for theimaging method shown in FIG. 3 are given, so as to omit duplicatedetailed description. In each step of imaging method shown in FIG. 3,identified area 44 is detected and protected area 46 is specified toapply protection to image data sets V10, V20, and V30 captured bycameras 11 and 12. In the protected reproduction method shown in FIG. 9,identified area 44 is detected and protected area 46 is specified toapply protection to image data recorded in recording medium 22.

As shown in FIG. 9, when the user inputs the start of protectedreproduction to camera device 10 via user I/F 24, an image designated bythe user is selected, and camera device 10 starts reproduction (StepS200). In other words, a reproduction step starts. When the reproductionstep starts, processing unit 19 obtains the image and audio informationand the image auxiliary information attached to the image and audioinformation from recording medium 22 (Step S202). The image and audioinformation includes the image data and the audio data. If subjects 30and 31 including a target to be protected are captured in an image to bereproduced, the subject information on a target to be protected isrecorded in recording medium 22 as the image auxiliary informationbelonging to the image and audio information. Accordingly, if no imageauxiliary information belonging to the image and audio information isrecorded in recording medium 22, an image to be reproduced is determinedto be not an image capturing a protection target. Therefore, processingunit 19 determines whether or not the image auxiliary informationattached to the image and audio information obtained is recorded inrecording medium 22 (Step S204). If no image auxiliary information isobtained (No), the operation returns to Step S202. If the imageauxiliary information is obtained (Yes), the operation proceeds to StepS206.

The image auxiliary information obtained by processing unit 19 isnotified to and analyzed in protection determination unit 71. Next, inthe protection determination step, protection determination unit 71analyzes the image auxiliary information obtained, and determines thepresence of information on copyright or privacy protection in the imageauxiliary information (Step S206). If protection determination unit 71determines that protection is required (Yes) based on the presence ofinformation on copyright or privacy protection, the operation proceedsto Step S114. If protection determination unit 71 determines thatprotection is not required (No), the operation proceeds to Step S202. Incase of proceeding to Step S202, processing unit 19 obtains the imageand audio information and the image auxiliary information again.

In the protection determination step, if protection determination unit71 determines that protection is “required,” as a result of analysis ofthe subject information included in the image auxiliary information,each step on and after the protection target information acquisitionstep is executed, same as that shown in FIG. 3. In the protected areacontrol step, protected area control unit 72 identifies a protected areabased on the subject information and the imaging condition informationincluded in the image auxiliary information.

After each step on and after the protection target informationacquisition step is executed, protection control unit 17 determineswhether or not the instruction information for completing protectivereproduction is notified from user I/F 24. If the instructioninformation for completing reproduction is notified (Yes), reproductionends (Step S230). If no instruction information for completingreproduction is notified (No), the operation proceeds to Step S202again, and the above steps are repeated.

As described above the protected reproduction method applied to cameradevice 10 of the present invention is a protected reproduction method ofreproducing an imaging target after applying protection to subjects 30and 31 to apply copyright or privacy protection. The protectedreproduction method at least includes the reproduction step, the imageauxiliary information acquisition step, the identified area detectionstep and the image data processing step. The reproduction step is toreproduce image data of an area captured within shooting field 40. Theimage auxiliary information acquisition step is to obtain the subjectinformation including the size, shape, direction and position of aspecific subject. This subject information is attached to image data.The identified area detection step is to dynamically detect identifiedarea 44 corresponding to specific subjects 30 and 31 included in theimage data obtained in the reproduction step. The image data processingstep is to specify protected area 46 from identified area 44 detected inthe identified area detection step, based on the subject informationobtained in the image auxiliary information acquisition step, and toapply protection to protected area 46 if subjects 30 and 31 requirecopyright or privacy protection. By executing these steps fromreproduction to image data protection, protected area 46 to applycopyright or privacy protection in an image in shooting field 40including subjects 30 and 31 to be protected is dynamically specifiedand protected on reproducing the image. In particular, since theposition and the size of protected area to be protected are calculatedfor each image frame, for example, so as to apply protection, identifiedarea 44 to be protected in a moving image is followed and protected. Inaddition, the size of protected area is accurately corrected by usingthe operation information of camera device 10 and the distanceinformation on a distance between camera device 10 and subjects 30 and31. Accordingly, even if the size of subject in the image changes due tomovement of subjects 30 and 31 to be protected or camera device 10 or achange in zooming rate, protection is applicable to the subjectcorresponding to any change in the size.

The present invention is described in accordance with the aboveexemplary embodiment. However, the present invention is not limited tothe above exemplary embodiment. The present invention further includesthe following structures.

A part of components configuring camera device 10 shown in FIG. 1 may beincluded in one piece of a system LSI (Large Scale Integration). Thesystem LSI is typically an ultra-multifunctional LSI manufactured byintegrating multiple components on one chip. If a part of components ofcamera device 10 is included in the system LSI, high-speed processingbecomes feasible by configuring at least image and audio forming unit16, protection control unit 17, protection processing unit 18 anddetection unit 83 with the system LSI. In addition, processing unit 13,communications unit 14, camera control unit 15 and memory 81 may also beintegrated in the system LSI. Still more, processing unit 19, R/Wcontrol unit 20 and output I/F 21 may also be integrated. Furthermore,distance estimation unit 82, extraction unit 84 and shape searching unit85 may also be integrated.

With respect to circuits realizing camera device 10 of the presentinvention, each of components configuring circuits may take the form ofa chip separately. Alternatively, a part or all components may beincluded in one chip. A one-chip structure enables the same structure asthe system LSI.

The present invention is not limited to the system LSI. The presentinvention may also adopt a structure called IC, LSI, super LSI or ultraLSI, depending on a degree of integration. In addition, circuitintegration is not limited to LSI. A dedicated circuit orgeneral-purpose processing unit is also applicable. In addition, PFGA(Field Programmable Gate Array) that allows programming aftermanufacturing LSI or a reconfigurable processing unit that allowsreconfiguration of connection and setting of circuit cells inside LSI isalso applicable.

Still more, if another integrated circuit technology replacing LSI isintroduced in line with advancement in semiconductor technology or othertechnology derived from advancement, it is apparent that functionalblocks may be integrated using new technology. Application ofbiotechnology is future possibility.

The imaging method of the present invention may also be a computerprogram operated typically by a computer. Still more, the imaging methodmay also be digital signals of computer program.

Still more the present invention may have a structure that a computerprogram or digital signal is recorded on readable recording medium suchas flexible disk, hard disk, CD-ROM, MO, DVD, DVD-ROM, DVD-RAM, BD andsemiconductor memory. Digital signals recorded in these recording mediaare also applicable.

Still more, the present invention may have a structure that a computerprogram or digital signal is transmitted via telecommunication line,wireless network, wire communication line and other networks typicallyInternet, or data broadcast.

Still more, the present invention may be a computer system including amicroprocessor and memory. The memory stores the computer program, andthe microprocessor operates according to the computer program.

Still more, the present invention may have a structure that anotherindependent computer system executes the operation by transmitting acomputer program or digital signal recorded on a recording medium, ortransmitting a computer program or digital signal via a network.

Furthermore, the present invention may have a structure in which theexemplary embodiment and other variations are combined.

Industrial Applicability

The camera device of the present invention follows and identifies asubject requiring copyright, portrait right or other privacyprotections, and applies protection to the image area of the subjecteven if the subject moves or changes. Accordingly, the present inventionis applicable to a digital camera that photographs and captures videodata or still-image data, and to an imaging method adopted in monitorcameras and other electronic devices.

The invention claimed is:
 1. A camera device comprising: a plurality ofimaging units for imaging an area in a shooting field so as to obtain animage data; a communications unit for communicating with a specificsubject and receiving object information including a size, shape,direction and position of the specific subject as subject information; adistance estimation unit for estimating distance information of thespecific subject with respect to the camera device by estimating asideway deviation of the specific subject using at least two of theplurality of imaging units; an identified area detection unit fordynamically detecting an identified area corresponding to the subjectincluded in the image data obtained by the imaging unit according to atleast the distance information; and an image data processing unit forspecifying a protected area in the identified area detected by theidentified area detection unit based on protection target informationincluded in the subject information received by the communications unit,and applying protection only to the protected area when the subjectincludes a protection target to be protected, the subject informationincludes both of the object information and the protection targetinformation including protection target size, protection target shape,protection target direction and protection target position related tothe protection target.
 2. The camera device of claim 1, furthercomprising: a user designation unit for designating the protected areaby user; wherein the image data processing unit specifies auser-specific protected area in the identified area based on theprotection target information included in the subject information, andapplies protection.
 3. The camera device of claim 1, wherein the imagedata processing unit includes a protection accuracy control unit forcontrolling accuracy of protection applied to the protected area.
 4. Thecamera device of claim 3, wherein the protection accuracy control unitcontrols protection depending on a direction of the subject, theprotection being applied to a projective area relative to a photographedface of an area requiring protection in the subject.
 5. The cameradevice of claim 3, wherein the protection accuracy control unit controlsprotection by adjusting a margin area around the subject to be protectedto a predetermined range and shape in accordance with at least one ofuser setting and use environment.
 6. The camera device of claim 1,wherein the image data processing unit includes a protection timecontrol unit for controlling time to apply protection to the protectedarea.
 7. The camera device of claim 6, wherein the protection timecontrol unit applies protection to the protected area in accordance withuser setting.
 8. The camera device of claim 6, wherein the protectiontime control unit switches protection applied to the protected areabetween a dynamic mask control and a static mask control, depending ontime, in accordance with user setting.
 9. A camera device comprising: animaging unit for imaging an area in a shooting field so as to obtain animage data; a communications unit for communicating with a specificsubject and receiving object information including a size, shape,direction and position of the specific subject as subject information;an identified area detection unit for dynamically detecting anidentified area corresponding to the subject included in the image dataobtained by the imaging unit; and an image data processing unit forspecifying a protected area in the identified area detected by theidentified area detection unit based on protection target informationincluded in the subject information received by the communications unit,and applying protection only to the protected area when the subjectincludes a protection target to be protected, the subject informationincludes both of the object information and the protection targetinformation including protection target size, protection target shape,protection target direction and protection target position related tothe protection target; wherein the image data processing unit includes aprotection time control unit for controlling time to apply protection tothe protected area; and wherein the subject includes an image displayunit reproducing and displaying a content to be protected, the imagedisplay unit holds EPG information, the communications unit obtains theEPG information, and the protection time control unit limits the time toapply protection to the protected area to time corresponding tobroadcast time of a specific program in accordance with the EPGinformation obtained.
 10. The camera device of claim 1, wherein theimage data processing unit includes a protected area control unit forcontrolling the protected area.
 11. The camera device of claim 10,wherein the protected area control unit changes the protected areafollowing one of movement of the subject and operation of the cameradevice.
 12. The camera device of claim 10, wherein the protected areacontrol unit changes the protected area following one of zoom-in andzoom-out operations.
 13. The camera device of claim 1, furthercomprising: a recording unit for recording the subject information onthe subject to which one of copyright and privacy protections is appliedas image auxiliary information that is separate from the imageinformation.
 14. The camera device of claim 1, wherein the image dataprocessing unit detects an overlapped portion of multiple subjects basedon the subject information obtained by the communications unit anddynamically changes the protected area when portions of the multiplesubjects are overlapped.
 15. The camera device of claim 1, wherein theimage data processing unit dynamically replaces the protected area ofthe subject with a different image.
 16. The camera device of claim 1,wherein the image data processing unit includes a protected areaprediction unit for predicting the protected area and preparing amasking area in advance, the protected area being predicted based on atleast one of a movement of the subject and information on a frame beforeor after a recorded or reproduced image, and the image data processingunit applying protection to the masking area prepared.
 17. The cameradevice of claim 1, further comprising: a gyro sensor, wherein the imagedata processing unit includes a protected area prediction unit forpredicting the protected area and preparing a masking area in advance,the protected area being predicted based on output information of thegyro sensor, and the image data processing unit applying protection tothe masking area prepared.
 18. An imaging method of applying protectionto a subject to be protected in an imaging target in an imagingoperation, the method comprising: an imaging step of obtaining an imagedata by imaging an area in a shooting field; a communications step ofcommunicating with the subject and receiving object informationincluding a size, shape, direction and position of the subject, thesubject information being provided with the subject as subjectinformation; a distance estimating step of estimating sideway deviationof the subject for estimating distance information; an identified areadetection step of dynamically detecting an identified area correspondingto a specific subject included in the image data obtained; and an imagedata processing step of specifying a protected area in the identifiedarea detected based on protection target information included in thesubject information received, and applying protection only to theprotected area when the subject includes a protection target to beprotected, the subject information includes both of the objectinformation and the protection target information including protectiontarget size, protection target shape, protection target direction andprotection target position related to the protection target.
 19. Thecamera device of claim 2, wherein the image data processing unitincludes a protection accuracy control unit for controlling accuracy ofprotection applied to the protected area.
 20. The camera device of claim2, wherein the image data processing unit includes a protection timecontrol unit for controlling time to apply protection to the protectedarea.
 21. The camera device of claim 2, wherein the image dataprocessing unit includes a protected area control unit for controllingthe protected area.
 22. The camera device of claim 2, furthercomprising: a recording unit for recording the subject information onthe subject to which one of copyright and privacy protections is appliedas image auxiliary information that is separate from the imageinformation.
 23. The camera device of claim 2, wherein the image dataprocessing unit detects an overlapped portion of multiple subjects basedon the subject information obtained by the communications unit anddynamically changes the protected area when portions of the multiplesubjects are overlapped.
 24. The camera device of claim 2, wherein theimage data processing unit dynamically replaces the protected area ofthe subject with a different image.
 25. The camera device of claim 2,wherein the image data processing unit includes a protected areaprediction unit for predicting the protected area and preparing amasking area in advance, the protected area being predicted based on atleast one of a movement of the subject and information on a frame beforeor after a recorded or reproduced image, and the image data processingunit applying protection to the masking area prepared.
 26. The cameradevice of claim 2, further comprising: a gyro sensor, wherein the imagedata processing unit includes a protected area prediction unit forpredicting the protected area and preparing a masking area in advance,the protected area being predicted based on output information of thegyro sensor, and the image data processing unit applying protection tothe masking area prepared.
 27. A camera device comprising: a pluralityof imaging units for imaging an area in a shooting field so as to obtainan image data; a communications unit for communicating with a specificsubject and receiving object information including an object size,object shape, object direction and object position, relating to thespecific subject; and protection target information including aprotection target size, a protection target shape, a protection targetdirection and a protection target position, related to a protectiontarget, a distance estimation unit for estimating distance informationof the specific subject with respect to the camera device by estimatinga sideway deviation of the specific subject using at least two of theplurality of imaging units; an identified area detection unit fordynamically detecting an identified area corresponding to the subjectincluded in the image data obtained by the imaging unit according to atleast the distance information; and an image data processing unit forspecifying a protected area in the identified area based on theprotection target information received by the communications unit, andapplying protection only to the protected area when the subject includesa protection target to be protected and the subject information includesboth the object information and the protection target information. 28.An imaging method of applying protection to a subject to be protected inan imaging target in an imaging operation, the method comprising: animaging step of obtaining an image data by imaging an area in a shootingfield; a communications step of communicating with the subject andreceiving at least object information including an object size, objectshape, object direction and object position, relating to the subject,the object information being provided with the subject as subjectinformation; a distance estimating step of estimating sideway deviationof the subject for estimating distance information; an identified areadetection step of dynamically detecting an identified area correspondingto a specific subject included in the image data obtained; and an imagedata processing step of specifying a protected area in the identifiedarea detected based on protection target information including aprotection target size, a protection target shape, a protection targetdirection and a protection target position, related to a protectiontarget , and applying protection only to the protected area when thesubject includes the protection target to be protected and the subjectinformation includes both of the object information and the protectiontarget information.
 29. The camera device of claim 1, wherein thedistance estimation unit estimates a first distance of the specificsubject with respect to the camera device according to a first estimatedsideway deviation being larger than a second estimated sideway deviationfor a second distance, wherein the first distance is shorter than thesecond distance.
 30. The camera device of claim 1, wherein the distanceestimation unit estimates the sideway deviation according to a portionof the image data captured by the at least two of the plurality ofimaging units.